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ABSTRACT 

The Communication and Theatre Arts Department at 
Heidelberg College (a private liberal arts college in northwestern 
Ohio) developed a departmental assessment plan. The department 
faculty began by working to establish "exit criteria 11 during an 
all-day meeting. Unlike other disciplines, communication has no 
standardized tests and performance measures which can easily be 
adapted to an institution's needs. Also, since Heidelberg College 
does not offer release time, all assessment activities had to be 
added to full loads of teaching and co-curricular activities. Given 
the difficulties of participation in a junior year pilot assessment 
program, the department elected to conduct most assessment activities 
in the required senior seminar course where participation could be 
mandated. The Watson Glaser Test of Critical Thinking was 
administered, as well as instruments to measure writing skills, 
speaking skills, and student attitudes. None of the first-year data 
from these measures was significant by itself nor did it reveal much 
collectively. Informal discussions of the process and initial 
attempts in assessment have changed the focus of the faculty's 
thinking about the program. An alumni survey was developed and 
administered. Only about 25% of the surveys have been returned so 
far. In papers written for the senior seminar, students indicated 
they are pleased with hands-on experiences and the range of 
opportunities both academic and co-curricular. Preliminary response 
to the assessment program is that the process may be more valuable 
than the results. (Contains 18 references and a list of exit 
criteria.) (RS) 
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The 1980's was the decade of talking about assessment; the 1990's, the decade of doing 
assessment. Patricia Thrash (1990), Executive Director of North Central's Commission on 
Institutions of Higher Education, termed assessment and accountability "the most important 
emphasis of the new decade." The impetus for this activity came from national reports critical 
of higher education cuch as the Association of American College's "Integrity in the Classroom" as 
well as from state and accrediting agency mandates (Backlund, Hay, Harper and Williams, 
1989). All six accrediting agencies now require some form of assessment as a condition for 
accreditation (Wolff, 1990). 

As the decade opened, El-Khawas (1990) found that 82 percent of American colleges and 
universities had some form of assessment activity underway. The ACE/Winthrop survey (Prus, 
El-Khawas, Anderson, Johnson and Cowart, 1990), however, revealed that only a third of the 
assessing institutions were operating a comprehensive program to measure student learning and 
development. Although another national survey found that a majority of communication 
departments were not engaged in assessment (Hay, 1992), more will begin or intensify 
activties as their accreditation time draws near. 

From the beginning of the movement, several questions have been particularly vexing. "What 
should we measure?" "What measures should we use to obtain the desired information?" "Are 
we looking at individual students or at programs?" "Should we start with the institution or the 
department?" This paper details the efforts of the Communication and Theatre Arts Department 
at Heidelberg College 1 to answer these and other questions as a departmental assessment pian 
was developed. 

Although those involved in assessment have always contended that the department is "an 
appropriate and powerful context for assessment" (Hutchins, 1989), even there the process is 
not without difficulties. While in large universities departmental assessment may be more 



'Heidelberg College is a private liberal arts college located in northwestern Ohio with an 
enrollment of about 1200. The Department of Communication has four full time and three part 
time faculty. Eighty students are enrolled in the single major which is divided into Theatre and 
Communication/Media tracks. 
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manageable than is institutional evaluation, the small college department also encounters a 
formidable array of problems. There is much the assessment literature does not mention. 

SIMPLISTIC ASSUMPTIONS 

In M. Scott Peck's words, assessment is difficult. Hay (1992) noted that communication 
educators face a variety of challenges. Soma of our courses are genera! education courses; some 
are required for the range of majors found in our departments. We develop a variety of skills, 
behaviors and attitudes which are difficult to measure with traditional measures. In small 
institutions and departments where there may be only two or three faculty members, we often 
function with heavy teaching and co-curricular loads. In addition we face several problems 
common to all assessors. Among these are simplistic assumptions about (1) goals, (2) 
measurement tools and (3) required time. 

Assessment by definition is measuring how well goals are met. The Denver Conference 
Program Assessment Group's First Principle (Hay, 1991) reads, "Assessment should be based 
on goals/objectives defined and operationalized by the faculty of specific programs within the 
context of a particular department." Smith and Hunt (1990) comment, "If nothing else, 
assessment has commanded that programs, departments and even entire institutions express in 
unequivocal terms who they are, what they do, and how they know when they have achieved it." 
Similarly, Kean College's Statement of Principles (Knight and Lumsden, 1990) describes the 
purpose of assessment in operational terms. An assessment program should: 

a) articulate the goals of each academic program 

b) gain feedback on the progress toward those goals 

c) use feedback to modify aspects of each academic program to ensure that goals 
are being achieved 

Thus, assessment programs assume that departments have clearly defined goals which drive 
the academic and co-curricular program. Yet for many, those goals are only implicitly 
understood. In a national survey, Hay (1992) learned that only 34% of the responding 



Assessment in the Capstone Course 



3 



departments had defined learning goals and objectives for their majors/minors. In 1989, 
Heidelberg would have been included in that group. During a day-long meeting in August of that 
year, departmental faculty worked to establish "exit criteria." Attempting to identify the 
characteristics we want our graduates to have, we found formulating lists of desired attitudes, 
values and skills time consuming but possible. Since that time, we have struggled to determine 
knowledge objectives for all majors and more specialized objectives for those students who take 
either the Communication/Media or Theatre tracks. Agreeing upon common and essential 
knowledge has been an on-going, major challenge. 

A second simplistic assumption is that once goals are set, doing assessment is easily completed. 
Unlike other disciplines there are no standardized tests and performance measures of 
communication which can be easily adapted to an institution's needs. This comes as no surpise to 
those in the discipline. Wartella (1993) in her ICA presidential address, described 
communication education as haphazard, with no canon, no core knowledge we can expect from 
undergraduates. Further, Pearson and Daniels (1988) pointed out that we do not agree on 
expected communication competencies. Attempts to develop nationally accepted measures have 
been made with Rubin's Communication Competence Assessment Inventory being the most well 
known but it is not used in all institutions. Nor do we necessarily agree on the interpretation of 
a given instrument. Those attending the Assessment Short Course at the SCA convention in 
Chicago (1992) will remember the spirited disagreements over the application of "The 
Competent Speaker" speech evaluation form. 2 

SCA's Principles of Assessment (Hay, 1991) reveal additional concerns. Principle 5 reads, 
"Assessment should be based on multiple methods appropriate to a given program rather than 
any single test." Principle 6 asserts, "Assessment instruments and procedures should be 
externally validated." Thus, SCA recognizes our different needs, suggests we develop our own 
measures and use external validation measures. For the small department in a small 

2 The short course was conducted by Sherwyn Morreale and Ruth Hulbert-Johnson both of the 
University of Colorado, Colorado Springs and by K. Phillip Taylor of the University of Central 
Florida, Orlando and Donna S. Tatum, University of Chicago. 
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institution, these may be almost insurmountable hurdles. Many will not have time, money 
and/or expertise to develop such measures. Indeed, 40% of the respondents to Hay's survey 
(1992) indicated that "their department was in need of measures and instruments to use in 
gathering information." 

The third common assumption is that these activities can be easily fitted into the time frames 
of communication faculty. For some departments, somewhere, that might be possible. But for 
many, defining objectives, gathering information and interpreting data is an additional, heavy 
burden which needs to be acknowledged at the outset. 

Beginning the process is especially time consuming. After our first one-day meeting, 
Heidelberg departmental faculty attempted to devote some faculty meeting time to further 
planning. With the press of regular problems this was very difficult, although as will be noted 
later, these sessions produced some of our best insights. When we held a half-day session in 
1993 to continue our work on objectives, members likened the experience to the Allen Alda 
movie, "Same Time Next Year." It had been a long time since we had last worked on objectives. 

An example of the required time commitment was detailed by Parker and Drummond-Reeves 
(1992) of Boise State University. They concluded that an entire year was necessary to develop 
an alumni survey. They further argued that one-quarter release time for a semester was 
inadequate to design and complete an alumni survey and recommended that release time span at 
least two semesters. Heidelberg does not offer released time for activities, so all assessment 
including an alumni survey must be added to full loads of teaching and co-curricular activities. 
Many other communication departments will probably be working assessment into already busy 
schedules. College officials mandating assessment need to be reminded of the large amounts of 
time required and be asked to support the process in some way. 

Having noted some of the difficulties and simplistic assumptions, we must do assessment. 
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TRIAL AND ERROR IN THE CAPSTONE COURSE 
An early, important decision is whether assessment will be formative or summative or both. 
According to Tucker (in press), formative assessment is that which provides feedback directly 
to the student. In the classroom, formative assessment is conducted at the end of a class or 
section of material. Institutionally, information may be collected at some point during the 
student's academic path so that corrective measures may be instituted. For example, a junior 
year assessment which reveals that a student has less than average critical thinking skills can 
help that student identify strategies to improve those skills during the final year of study. Such 
information can also be used tor program improvement. 

Summative assessment is the collection of information at the end of the collegiate career and 
is used primarily for program improvement. Tucker (in press) points out that this 
information is "not nearly as close to the problem and is subject to a myriad of intervening 
variables." 

Formative Measures Used at Heidelberg 

In 1990 and 1991, a junior year appraisal program was conducted by Wolff as a pilot for a 
possible institutional model. In that program, first semester junior communication and theatre 
arts majors were asked to voluntarily take part in several assessment activities. Among these 
was the creation of a portfolio to demonstrate writing and critical thinking skills. 
Demonstrations of ".Uitudes and participation in the fine arts and the students' understanding of 
other cultures were also to be included. The Watson-Glaser Test of Critical Thinking was 
administered and students were asked to develop a functional resume 3 , and submit a videotape of 
an oral presentation. After the materials were submitted and evaluated, a student-faculty 
conference led to the creation of a goal plan for the student's final three semesters 4 . 

3 This resume emphasizes skill development more than educational and work experiences. The 
resume serves as a self-report of student perception of the development of skills. 
4 At Heidelberg, students learn to write goal plans during the freshman Total Student 
Development program. They are urged to keep these updated during their academic career. 
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The junior year appraisal was highly valued by participating students. Getting students to 
participate and to collect materials even after entering the program was, however, very 
difficult. Evaluating the portfolios also proved to be more difficult than anticipated. Listening 
to portfolio devotees at national meetings made the process sound easy, but the need to create 
standards which could then be tested for reliability was a task requiring both expertise and 
large amounts of time. Administering and interpreting the program took inordinate amounts of 
time. While the approach provided useful feedback to the student and the department, it was 
deemed unworkable for both the department and the college as a whole. 

Other formative assessment strategies, including Minute Papers and Muddiest Point, have been 
used on occasion by departmental faculty. Angelo and Cross's (1993) informal classroom 
assessment procedures are well described by Tucker (in press), as are a variety of classroom 
research strategies. These have much merit but systematic classroom research has not been 
undertaken in the department. 

Formative techniques are most useful to faculty and students, providing the opportunity for 
corrective responses to less than desirable situations. It is unclear, however, if they will be 
acceptable and/or sufficient to meet the accrediting agencies' mandates. Relying on these alone 
would be risky even though they could, and should, be used in combination with summative 
measures. 

Summative Measures Used at Heidelberg 

Given the difficulties of participation in the Junior Year Appraisal Program, the department 
elected to conduct most assessment activities in the required Senior Seminar course where 
participation could be mandated. The course was chosen because we consider it a capstone 
experience in which we ask students to demonstrate skills of analysis, synthesis, writing and 
speaking. In place for over 25 years, the course has evolved into a study of communication 
criticism where students produce a critical study in both written and oral formats. 
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Northeast Missouri State University has been using capstone course assessment for several 
years. They report the following: 

Capstone courses in the major field integrate both university-wide general education 
requirements and subfields of the discipline. They also provide a systematic focus 
on knowledge, skills and attitudes outcomes; create opportunities for multiple 
assessment; prompt faculty discussion; and encourage reexamination of discipline 
requirements and teaching approaches. {Update, November-December, 1992) 
The first step in developing the assessment plan was to identify strategies to evaluate the 
department's skills and attitudes objectives. The assessment programs of several institutions 
were used for guidance. Among these were Northeastern Missouri State University 
(Heisserer), Kean College, and King College. Goals and selected strategies are included in 
Appendix A. Plans were then made for implementing some of the strategies. 

In the first year (1992-93), the department decided to assess primarily critical thinking, 
writing and speaking skills. As two faculty members were also working on the institutional 
subcommittee to evaluate speaking and writing, the department elected to pilot that committee's 
evaluation instruments. Criteria for speaking were selected from SCA's Communication is Life 
(1990) and from the college's speech evaluation form. Writing criteria were written by 
members of the English Department. Both sets were submitted to the general faculty for 
comment and instruments to evaluate student achievement were drawn up. 

A student survey to determine student participation in and attitudes about the department's 
co-curricular and social activities was also created and administered. 
During the fall semester, we engaged in the following formal assessment activities: 
•The Watson Glaser Test of Critical Thinking 5 was administered. 

Critical thinking was also evaluated as part of the regular grading in the course. 
•Writing skills were evaluated using the pilot instrument of the Writing and 

5 The Watson-Glaser Test was used because the College is beginning to collect entry level Watson 
Glaser scores so that in 1994-95, it will be possible to do individual pretest-posttest. In 
addition, by using the same tool as the rest of the college (whatever the limitations of that tool), 
across campus comparisons will also be possible. 
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Speaking Subcommittee of the college's Evaluation Taskforce. 
•Speaking skills were evaluated using the pilot instrument of the Writing and 

Speaking Subcommittee of the college's Evaluation Taskforce. 
•A survey of student participation in and evaluation of departmental activities was 

administered. 

Results of Initial Assess ment Strategies 
Evaluation of the information followed in the second semester. The following data was obtained: 
•The Watson Glaser test results when compared to the norms for upperclass in four year 
colleges indicated a bi-modal distribution of scores. Three students whose scores were 
below the 50th percentile had high GPA's. This is a flag to both the department and the 
college. 

♦The instruments for evaluating speaking and writing were unsatisfactory and yielded 
little information beyond that obtained informally in regular grading. Newer, simplified 
instruments will be developed and tested before the completion of 1993-94 seminar. 
•Administration of the participation and attitude survey indicated that students were 
generally supportive of departmental social activities such as the fall dessert and the 
spring awards banquet. Participation in co-curricular activities was linked to the 
student's track (theatre students participated in and attended theatre productions while 
Communication/Media track majors tended to be more active in radio and television.) 
Only a small percentage of the students had completed the required speech competition 
requirement. 

None of this data was significant by itself nor did it reveal much collectively. The first trial did 
establish the beginnings of a data base for the Watson Glaser and Student Survey results. The 
trial use of the writing and speaking instruments, while not productive for the department, 
provided a se vice for the institution. As the college begins to collect entry level data on these 
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skills (probably in 1994-95), using the same instrument will allow us to compare our 
students with those in other majors. Until then, we will be able to compare across years. 

In short, formal assessment the first year didn't tell us much we didn't know. We started and 
learned what not to do in several instances. As assessment continues in 1993-94,. computer 
files are being established to facilitate comparison of yearly data. 

Informal Assessment 

Informal assessment has been perhaps more significant. Discussions of the process and initial 
attempts to do assessment have changed the focus of the faculty's thinking about our program. 
For example, in one session, the faculty considered the ability of students to raise and handle 
questions. In a discussion session with the seminar members, students raised similar concerns. 
Several found the questions following their prospectus presentations 6 threatening while others 
considered it a learning experience. As a result of these sessions, the faculty recognized that we 
do not systematically teach handling of questions. 

Further, these informal sessions have led the faculty to acknowledge that while there is 
agreement on the core curriculum required in each track, we have only sketchy knowledge of 
what is actually taught in courses other than our own. Teaching of skills is generally apparent 
but a comprehensive plan for identifying where concepts are taught has never been created. 
This accounts in part for our difficulty in identifying knowledge objectives. 

Assessment in 1993-94 

During the summer and early fall, an alumni survey was developed. The objectives of the 
survey were these: 

1. to learn alumni evaluation of the academic program, including specific courses 

2. to learn alumni evaluation of the co-curricular program 

6 As a part of the seminar, all students write a prospectus of their study. These are presented 
orally before the class and the departmental faculty. Students stand for questions after the 
presentation. 



eric 



11 



Assessment in the Capstone Course 1 0 



3. to determine the career paths of graduates 

4. to assess graduates' attitudes of the department's role in skill building, particularly 

speaking, writing and critical thinking 

5. To obtain alumni reaction to the creation of the two tracks (1991) 

In developing the survey, the department consulted earlier surveys done by the department and 
the college, the survey prepared by the Computer Science Department for assessment purposes 
(Van Vleet), and the work of Parker and Drummond-Reeves (1992) of Boise State University. 

Although the advantages of a telephone study are many, factors of cost and available time led to 
a mail survey. The instrument was sent to all graduates of the past ten years, a total of 127 
persons. 

Students in the seminar were asked to write a paper in which they identified concepts which 
they believe to be key to their four year study in communication and theatre. This is similar 
to the Senior Letter activity used at Karen Wilson College (McKinley and McWilliams, 1992). 
There all seniors submit a letter requesting graduation. Students provide a global survey of 
educational activities and evaluate the general outcomes of their education. They are asked to 
comment on what has been of especial value, their contributions to the college and education and 
what recommendations they might make for the future of the college. 

After writingtheir concept papers, our students were invited to a discussion meeting with the 
faculty. The essays and meeting notes were analyzed and compared to the knowledge objectives 
which the faculty has identified at this point. We had hoped this would give us significant 
insight into our effectiveness in teaching concepts. 

Writing, speaking and critical thinking will be evaluated when students present their final 
papers and make the oral presentations. 

Second Year Assessment Results 

• We have about a 25% return for the alumni survey; a second reminder has been 
mailed. First reading suggests that those students who rate our program highly are 
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those responding. We hope some of the not-so-positive also comment. 

•Results of the student surveys are almost identical to the first year. 

•The second administering of the Watson-Glaser Test of Critical Thinking again revealed 

a bi-modal spread of scores. It is becoming apparent that the curriculum will need to 

be modified to include more overt development of critical thinking skills. 

•The senior essay and meeting indicated that students are pleased with the hands-on 

experience opportunities and the range of opportunities both academic and co- 

curricular. They expressed concern for the courses required in each track, generally 

supporting a broader curriculum. Students identified a problem concerning the public 

reporting of internships which will be altered. 

•In geneial, students did not respond to the request for "significant concepts learned." 
We need to work on the assignment details if this is to work as a measure of knowledge. 



CONCLUSION 

The process has been challenging, time consuming, rewarding. In spite of many models of 
assessment available, we have found that in many ways we must reinvent the wheel as we focus 
on the effectiveness of our particular program. Has all of this been worth the time? We don't 
know yet. Our preliminary response is that the process may be more valuable than the results. 
Much of what we've learned about the program and our teaching, we have at least suspected. But 
the process of setting objectives, gathering and analyzing data has provided an opportunity to 
focus our attention on what it is we do and how we do it. That may be the most valuable result of 
the exercise. 
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